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EXECUTIVE SUMMARY 


The Wallace Foundation’s Principal Supervisor Initiative (PSI) was a four-year effort to 
redefine principal supervision in six urban school districts. Launched in 2014, the PSI aimed to 
help these districts transform the principal supervisor position from a role that traditionally 
focused on managerial tasks to one dedicated to developing and supporting principals to be 
effective instructional leaders. The PSI encouraged these districts to revise the principal 
supervisors’ job description to focus on instructional leadership, reduce principal supervisors’ 
span of control (the number of principals they oversee), train supervisors and develop their 
capacity to support principals, create systems to identify and train new supervisors, and 
strengthen central office structures to support and sustain changes in the principal supervisor’s 
role. 


At the same time, other urban districts also were revising their approaches to principal 
supervision, prompted by national conferences, new principal supervision standards, and other 
local efforts (Cochran et al. 2020). For example, in 2015, the Council of Chief State School 
Officers released the first national standards for principal supervisors (Council of Chief State 
School Officers 2015). These standards set new expectations for the principal supervisor role, 
focusing on supporting principals as instructional leaders. 


This report compares principal supervision in the PSI districts with that in other urban 
districts. It compares (1) the structures districts put in place to support the principal supervisor 
role (including principal supervisors’ span of control and districts’ approaches to selecting and 
training supervisors) and (2) principal supervisors’ professional practices. This comparison can 
provide greater context for the changes that the PSI districts made during the initiative, 
highlighting areas where the PSI districts’ work aligned with broader national trends and areas 
where it surpassed or lagged these trends. It can also provide a roadmap for other districts 
seeking to strengthen the principal supervisor role to better support principals as instructional 
leaders, by highlighting aspects of principal supervision that districts and states may find easier 
to change and aspects that may require more deliberate action and focus. 


A. Methods 


This report presents findings from a 2018 survey of principal supervisors in 54 urban school 
districts, including the 6 PSI districts. The districts were all members of the Council of the Great 
City Schools, a coalition of the nation’s largest urban public school systems. In total, 343 
principal supervisors responded to the survey, including 50 in the 6 PSI districts (a 96 percent 
response rate) and 293 in the 48 other urban districts (a 64 percent response rate). 


B. Findings 


The study found several key differences, along with some important similarities, across 
principal supervisors in the PSI and other urban districts. In general, we found that the PSI 
districts implemented stronger structures than other urban districts to support principal 
supervisors in their work with principals, but that supervisors’ practices were similar across PSI 
and other urban districts. 


LEADING THE CHANGE: A COMPARISON OF PSI AND OTHER URBAN DISTRICTS VANDERBILT UNIVERSITY / MATHEMATICA 


Structures to support the principal supervisor role 


The PSI encouraged districts to make several structural changes to support the revised 
principal supervisor role, including reducing principal supervisors’ span of control, 
implementing training and mentorship for supervisors, creating succession and apprenticeship 
programs for aspiring supervisors, and creating structures in the central office to support 
principal supervisors in their work with principals. Consistent with the PSI’s emphasis on these 
changes, we found that the structures to support supervisors in their work with principals were 
stronger in PSI districts than in other urban districts. 


Gp Principal supervisors in PSI districts had lower spans of control than those in other 
urban districts. Reducing principal supervisors’ span of control was a key goal of the PSI, 
intended to enable supervisors to spend more time working with each of their principals. By 
the end of the initiative in the 2017-2018 school year, supervisors in PSI districts oversaw 
an average of 13 principals, compared with an average of 16 for principal supervisors in 
other urban districts. However, spans of control in both sets of districts were considerably 
lower than the average span of control of 24 in urban districts in 2012 (Casserly et al. 2013), 
suggesting that both PSI and other urban districts made structural changes to better support 
the principal supervisor role. 


@ Principal supervisors in PSI districts were more likely than those in other urban 
districts to oversee groups of principals organized by grade level. Same-grade-level 
groups can be advantageous in that they allow supervisors to focus on a narrower set of 
content standards, curricula, and assessment data. Principal supervisors in PSI districts were 
more likely to report that their principals were grouped based on grade level. In contrast, 
supervisors in other urban districts were more likely to indicate that their principals were 
grouped based on attendance feeder patterns (groups of elementary, middle, and high 
schools serving the same students over time), which required supervisors to work with 
principals across the K-12 spectrum. 


© Supervisors in PSI districts were more likely than those in other urban districts to 
receive role-specific training and rated their training more highly. Role-specific, 
dedicated training for principal supervisors was more prevalent in PSI districts than in other 
urban districts. Furthermore, although the trainings emphasized similar skills across PSI and 
other urban districts, supervisors in PSI districts rated the quality of their training more 
highly than did those in other urban districts. 


(4) PSI districts were more likely than other urban districts to offer programs for new and 
aspiring supervisors. Principal supervisors in PSI districts were significantly more likely 
than those in other urban districts to report that (1) their district had a program to train 
aspiring supervisors, (2) their district had a mentoring and induction program for new 
supervisors, and (3) they themselves had entered their position through participation in a 
program for aspiring principal supervisors. 


© Principal supervisors in PSI and other urban districts had similar perceptions of 
central office support and structures. Overall, fewer than half of principal supervisors in 
both PSI and other urban districts agreed that the central office understood their work, 
facilitated their work with principals, was organized to support principals, or scheduled 
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meetings to maximize their time in schools. However, supervisors in PSI districts were more 
likely than those in other urban districts to agree that they were involved in the deployment 
of instructional support staff to their schools. This involvement helped principal supervisors 
in the PSI districts direct central office support to schools based on each school’s specific 
needs. 


© Principal supervisors in PSI districts had more favorable views of their own evaluation 
systems than did those in other urban districts. Principal supervisors in the PSI districts 
rated the quality of their district’s supervisor evaluation systems higher than did those in 
other urban districts in terms of clarity of the evaluation process, usefulness of feedback, and 
alignment of the evaluation with their work. 


Principal supervisors’ professional practices 


The PSI aimed to change supervisors’ daily work and practices, particularly their work to 
support principal leadership. As the supervisor role shifted from a focus on management and 
compliance toward developing principals’ instructional leadership, supervisors were expected to 
adopt the mindsets and practices necessary to support principals. These practices included 
visiting schools, coaching and providing feedback to principals, conducting structured classroom 
walkthroughs and observations with principals, modeling and role-playing leadership behaviors, 
and guiding principals in analysis of and reflection on their school’s program of teaching and 
learning. Despite the PSI’s emphasis on these practices, we found that, in general, supervisors’ 
practices were similar across the PSI and other urban districts. 


1) Principal supervisors in both PSI and other urban districts spent much of their time 
visiting schools and focusing on instructional leadership. On average, supervisors in both 
sets of districts reported spending about half of their time on the job visiting schools and 
working with principals on instructional leadership. However, principal supervisors in PSI 
districts made more frequent visits to both their most- and least-visited schools than did 
supervisors in other urban districts, although these differences were not statistically 
significant. This may have been a result of their smaller spans of control. 


@ Principal supervisors in both PSI and other urban districts used similar instructional 
leadership practices with principals, although supervisors in PSI districts were less 
likely to work on operational issues with principals. Principal supervisors in both PSI and 
other urban districts used instructional leadership practices with principals, such as visiting 
classrooms and analyzing data, with similar frequency. However, consistent with the PSI’s 
attempts to shift supervisors away from operational issues, supervisors in PSI districts 
focused less on issues such as hiring and facilities than did supervisors in other urban 
districts. 


© Principal supervisors in PSI districts had more positive views of their districts’ 
principal evaluation systems than did those in other urban districts. The PSI districts 
focused on aligning their principal evaluation systems with supervisors’ work to support 
principals. Compared with principal supervisors in other urban districts, supervisors in PSI 
districts expressed more positive perceptions of their districts’ principal evaluation systems 
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and were more likely to report that the systems aligned with their work, although the latter 
difference was not statistically significant. 


Cc. Summary 


The PSI focused on developing a new role for principal supervisors as drivers of 
instructional leadership among principals and creating the district context to support the new 
vision of principal supervision. At the same time as the PSI unfolded, districts across the United 
States also were rethinking principal supervision, prompted by national conferences and 
workshops, new principal supervision standards, and other local efforts. However, the more 
intensive, sequential approach the PSI districts used to shift the focus of their principal 
supervisors’ work may have led to some changes above and beyond those in other urban 
districts. 


The biggest differences between PSI and other urban districts were in the structures to 
support the principal supervisor role. Compared with other urban districts, the PSI districts had 
lower spans of control, were more likely to provide unique and dedicated training for 
supervisors, and were more likely to offer programs for aspiring supervisors. In contrast, 
principal supervisors in PSI and other urban districts had similar perceptions of the support they 
received from the central office. Supervisors’ practices with their principals also were similar 
across PSI and other urban districts—supervisors reported spending more than half their time on 
instructional leadership and visiting schools. 


Our finding of similar supervisor practices across PSI and other urban districts is consistent 
with two other recent studies, one focused on PSI districts and one on a broader set of urban 
districts, that examine how principal supervisors’ work with principals has evolved. The first 
study, Goldring et al. (2020), found that supervisors’ work with principals in PSI districts 
changed over the course of the PSI. Principals reported increased frequency of supervisor 
practices such as coaching, feedback, and data use. Principals’ perceptions of their supervisor’s 
effectiveness improved over the course of the initiative as well. Principals recounted how, over 
the course of the initiative, their relationships with their supervisors had improved and deepened 
because supervisors better understood their contexts and specific needs. The second study, 
Cochran et al. (2020), examined trends in principal supervision in a broader set of urban districts. 
It suggests that principal supervisors in urban districts have changed their practices to focus more 
on instructional leadership in recent years. Together these findings suggest that it is possible for 
districts to change the principal supervisor role to better support principals. 


Deep changes to the principal supervisor role require clearly articulated expectations for the 
focus of the supervisor role, including strong expectations for consistency of practices; alignment 
with central office culture and capacity; and continued opportunities for principal supervisors to 
learn and refine practices. District leaders, practitioners, and researchers should continue to 
actively identify how and under what circumstances districts can promote these conditions to 
strengthen principal supervisors’ knowledge and practices. 
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I. INTRODUCTION 


The Wallace Foundation’s Principal Supervisor Initiative (PSI) was a four-year effort to 
redefine principal supervision in six urban school districts. Launched in 2014, the PSI aimed to 
help districts transform a position that traditionally focused on managerial tasks to one dedicated 
to developing and supporting principals to be effective instructional leaders. 


The theory of action for the PSI 
emphasized two types of changes. 
First, districts made structural changes Six districts participated in the PSI: 
to create the context and conditions Baltimore City Public Schools, Maryland? 
needed to change the principal 
supervisor role and supervisors’ 
professional practices. These changes 
included revising the official job 
description for principal supervisors, 
reducing the number of principals each 6 
supervisor oversaw, creating 


Broward County Public Schools, Florida 


. Des Moines Public Schools, lowa 


1. 
2. 
3. Cleveland Metropolitan School District, Ohio 
4 
5 


. Long Beach Unified School District, California 


. Minneapolis Public Schools, Minnesota 


: : Fae 4 DeKalb County School District (Georgia) was an initial participant in the PSI 
apprenticeship Prop ans for ocenine but withdrew after the first year, following a change in district leadership. 
supervisors, and training supervisors Baltimore City Public Schools joined the initiative after DeKalb withdrew. 


to better support principals. The PSI 
also restructured the central office to better support supervisors’ work with principals. Second, 
through these structural changes, the PSI districts focused on shifting principal supervisors’ 
mindsets and practices from management to coaching and support. Districts expected supervisors 
to implement new professional practices such as coaching principals, conducting walkthroughs, 
and observing classrooms. They also tried to align their principal evaluation systems to support 
supervisors in their work with principals. 


The PSI had five core components to guide districts’ changes to the principal supervisor 
role, but it gave districts leeway to interpret, adapt, and implement each component according to 
their local context and needs. The five components were: 


1. Revising the principal supervisors’ job description to focus on instructional leadership 


2. Reducing principal supervisors’ span of control (the number of principals they oversee) and 
changing how supervisors are assigned to principals 


Training supervisors and developing their capacity to support principals 
4. Developing systems to identify and train new supervisors (succession planning) 


Strengthening central office roles to support and sustain changes in the principal 
supervisor’s role 


By the time of the launch of the PSI in 2014, urban districts around the country also were 
considering changes to the principal supervisor role (Cochran et al. 2020). For example, in 2015, 
the Council of Chief State School Officers released the first national standards for principal 
supervisors (Council of Chief State School Officers 2015). These standards set new expectations 
for the principal supervisor role and focused on supporting principals as instructional leaders. 
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Technical assistance providers, such as the New York City Leadership Academy, began offering 
programs for principal supervisors. Other districts and foundations hosted leadership summits 
and workshops for principal supervisors. Today, principal supervision has become a profession 
distinct from that of other central office administrators. 


This report, “Leading the Change: A Comparison of the Principal Supervisor Role in 
Principal Supervisor Initiative Districts and Other Urban Districts,” is one of three reports 
sponsored by The Wallace Foundation that examine districts’ experiences with the PSI and its 
effects on principals’ performance. In our first and second reports—“A New Role Emerges for 
Principal Supervisors: Evidence from Six Districts in the Principal Supervisor Initiative” 
(Goldring et al. 2018) and “Changing the Principal Supervisor Role to Better Support Principals: 
Evidence from the Principal Supervisor Initiative” (Goldring et al. 2020)—-we investigated the 
PSI districts’ progress during the four years of the initiative. We found that all six PSI districts 
revised their principal supervisor job descriptions, reduced principal supervisors’ span of control, 
and provided supervisors with role-specific training. In addition, some districts addressed 
succession planning by creating apprenticeship programs for aspiring supervisors. Some districts 
made greater strides than others in strengthening central office supports for principal supervisors’ 
work with principals, the most challenging component for districts to implement. 


The goal of this report is to compare principal supervision in the PSI districts with that in a 
sample of other urban districts to provide greater context for the changes that the PSI districts 
made during the initiative. It compares two broad aspects of principal supervision: (1) districts’ 
structures to support a redesigned role for principal supervisors and (2) principal supervisors’ 
roles and practices (Figure I.1). Specifically, the report compares the districts’ structures to 
support the principal supervisor role, including supervisors’ span of control, supervisor selection 
and training, and supervisors’ perceptions of the support they received from the central office. It 
also compares principal supervisors’ daily work and practices and their perceptions of how well 
their districts’ principal evaluation systems aligned with their work with principals. The findings 
offer insights into the challenges districts face when revising the principal supervisor role to 
better support principals as instructional leaders. 


Figure 1.1. Two broad aspects of principal supervision 


Pr 


Principal supervisors’ 
roles and practices 


¢ Time in schools 


¢ Instructional leadership practices 
Vitam eda tares| ex= 16s) 


¢ Principal evaluation 
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ll. METHODS 


Teams of researchers at Vanderbilt, Mathematica, and the Council of the Great City Schools 
collaborated to design and administer the surveys to all principal supervisors in the PSI districts 
and the council’s other member districts in spring 2018. 


A. Instrument design 


The team adapted the survey instrument from those used in previous years of the PSI study 
(Goldring et al. 2018 and 2020) and an earlier study of principal supervision by the Council of 
the Great City Schools (Casserly et al. 2013).' Researchers at Vanderbilt reviewed, piloted, and 
revised the original PSI surveys to ensure their face validity and clarity. 


The survey asked supervisors about the following topics: 
Their professional background and demographics 


Their span of control (the number of principals they oversaw) and how these principals were 
assigned to them 


How they allocated their time 

Their practices and topics of discussion with principals 
Their professional development experiences 

Their perceptions of the central office 


Their perceptions of their own evaluations 


Their feedback concerning the supports supervisors need to be effective 


B. Survey administration, response rate, and final sample 


The study team sent the web-based surveys to all principal supervisors (staff directly 
overseeing one or more principals) in 63 participating Council of the Great City Schools member 
districts, including the 6 PSI districts. Seven other districts that were members of the council did 
not provide a list of their principal supervisors to the study team and were not included in the 
survey.” We administered the survey between April and September 2018. 


In 4 of the original 63 districts that received surveys, no supervisors responded. We also 
excluded five other districts from the analysis sample. Two of these districts—District of 
Columbia Public Schools and Tulsa Public Schools—were engaged in separate but concurrent 


' The final survey for this study is available online at https://www.wallacefoundation.org/knowledge- 
center/Pages/Leading-the-Change-A-Comparison-of-the-Principal-Supervisor-Role.aspx. 


> The Council of the Great City Schools had 70 member districts at the time of survey administration in 2018 and 76 
member districts when this report was published in 2020. 
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principal supervision changes with support from The Wallace Foundation. The other three— 
Charlotte-Mecklenburg Schools, Denver Public Schools, and Hillsborough County Public 
Schools—participated in the Principal Pipeline Initiative (PPI), a separate Wallace Foundation 
effort that also had implications for the principal supervisor role (the other three PPI districts 
were not members of the Council of the Great City Schools). Excluding these districts allowed us 
to compare supervisors in PSI districts with supervisors in urban districts that were not 
participating in intensive, targeted work that included some of the PSI’s approaches and 
components. Although the other urban districts in our sample may also have made changes to 
their principal supervisor role during the same period, these districts likely implemented such 
changes on their own, without external support. 


The final sample for this analysis included 343 confirmed principal supervisors across 54 
districts (a 67 percent response rate; Table II.1).* This included 50 supervisors in the 6 PSI 
districts (a 96 percent response rate) and 293 supervisors from 48 other urban districts (a 64 
percent response rate). The lower response rates in the other urban districts means that the 
responses in those districts might not be fully representative of the views of all the principal 
supervisors in those districts. This could lead us to overstate or understate the differences 
between supervisors in PSI and other urban districts. However, we are unable to determine the 
size or direction of any possible bias. 


Table II.1. Response rates and total respondents for PSI and other urban 
districts 


Response rate 


District type Number of districts Number of respondents (percentage) 
PSI 6 50 96 
Other urban 48 293 64 
Total 54 343 67 
Note: Table and analysis exclude principal supervisors in districts participating in The Wallace Foundation’s 


Principal Pipeline Initiative. They also exclude District of Columbia and Tulsa Public Schools, which were 
engaged in separate but concurrent principal supervision changes with support from The Wallace 
Foundation. They also exclude four respondents who we confirmed were not working in a principal 
supervisor role at the time of survey administration. 


Principal supervisors in PSI and other urban districts had similar professional backgrounds. 
The average principal supervisor in the PSI districts had spent about five years in the role, 
compared with six years for supervisors in other urban districts. In both PSI and other urban 
districts, most principal supervisors worked as principals immediately before becoming 
supervisors. However, PSI districts were more likely than other urban districts to hire their 
supervisors from principal positions in the district. 


> Based on an initial review of the survey responses, we determined that four respondents were not principal 
supervisors and dropped them from the analysis sample. We have also omitted one partial response from the PSI 
sample. 
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C. Analysis 


Our analyses compare the survey responses of principal supervisors in PSI and other urban 
districts. All analyses give equal weight to each principal supervisor, rather than averaging their 
responses and giving equal weight to each district. We assessed the statistical significance of 
differences in means between PSI and other urban districts using unpaired t-tests. In general, in 
our discussion of the findings, we focus on differences between PSI and other urban districts that 
are both substantively meaningful and statistically significant at the 5 percent level. We assess 
whether a difference is substantively meaningful based on context from our analysis of districts’ 
implementation of the PSI across the four years of the initiative (Goldring et al. 2018 and 2020). 
In discussing differences that are substantively meaningful but not statistically significant, we 
note the lack of statistical significance in the text. 
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lil. STRUCTURES TO SUPPORT A REDESIGNED ROLE FOR PRINCIPAL 
SUPERVISORS 


The PSI asked districts to make several structural changes to help transform the principal 
supervisor role. These changes entailed reorganizing, reallocating, and refocusing essential 
components and functions of the central office. The districts focused on redefining the official 
job description, reducing the span of control for principal supervisors, implementing training and 
mentorship programs for sitting principal supervisors, creating apprenticeship programs for 
aspiring supervisors, and revising principal supervisor evaluation systems. These changes were 
intended to create the conditions for supervisors to spend more time in schools and give them the 
knowledge and skills to implement specific supervisory practices to develop and support 
principals. 


A. Span of control and network assignments 


A major component of the PSI was reducing each supervisor’s span of control—the number 
of principals each supervisor was directly responsible for supporting. The theory behind this 
change was that, when principal supervisors oversee fewer principals, these supervisors can 
provide more individualized support to their principals, develop stronger relationships with their 
principals, and spend more time in schools. 


In addition to reducing the span of control, PSI districts grappled with how best to group or 
assign principals to networks and supervisors to maximize the effectiveness of supervisors’ 
support and principals’ learning opportunities with other principals in their networks. 


Span of control 


Reducing principal supervisors’ span of control requires substantial planning and central 
office reorganization. Districts must have the resources and stakeholder buy-in to hire more 
supervisors or to shift the responsibilities of existing central office staffers to full-time principal 
supervisor roles. Because of frequent pressure to allocate resources directly to schools and 
classrooms, it can be challenging for districts to increase the number of principal supervisors. 


By the end of the PSI, principal supervisors’ span of control was lower in the PSI districts 
than in the other urban districts (Figure III.1). Supervisors in PSI districts oversaw 13 principals 
on average, compared with an average of 16 principals for supervisors in other urban districts. 
The range of principals overseen by supervisors was also narrower in PSI districts. The span of 
control in PSI districts ranged from 7 to 19, compared with a range of 2 to 50 principals in other 
urban districts. However, spans of control in both sets of districts were considerably lower than 
the average span of control of 24 in urban districts in 2012 (Council of the Great City Schools 
2013), suggesting a broader national trend toward these reductions. 
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Figure 111.1. Principal supervisor span of control in PSI and other urban districts 
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Figure reads: The average supervisor in a PSI district oversaw 13 principals. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: Survey question read, “How many principals do you currently supervise?” Differences between supervisors 
in PSI and other urban districts are statistically significant at the 5 percent level. 


Consistent with the smaller average span of control in PSI districts, supervisors in these 
districts were less likely to report that they supervised too many principals. 


Supervisors with higher spans of control (defined as 13 or more principals) in both PSI and 
other urban districts were more likely to agree that they supervised too many principals to 
provide enough support (Figure II.2). However, interviews with supervisors in PSI districts 
(Goldring et al. 2020) suggest that even a modest reduction in span of control can substantially 
shape principal supervisors’ work and principals’ experience of support. Supervisors in PSI 
districts who experienced smaller reductions (for example, from 19 to 13 principals) said these 
changes allowed them to expand their focus to work with all their principals, even those who 
may have needed less support. 


In the PSI districts, supervisors, principals, and central office staff universally praised the 
reductions in span of control (Goldring et al. 2020). In interviews, central office personnel in the 
PSI districts said they felt that principals were better supported because they had greater access 
to their supervisors and more regular visits and contacts. Supervisors reported that a smaller span 
allowed them to understand their schools better and visit more frequently. Many principals also 
noted that they saw their principal supervisor more frequently or received more support after the 
decrease in span. 
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Despite the smaller average spans of control in PSI districts, however, supervisors in those 
districts still had commitments or faced obstacles that limited the time they could spend in 
schools or working directly with principals. In both PSI and other urban districts, nearly 40 
percent of supervisors with lower spans of control agreed or strongly agreed that they did not 
have time to visit certain schools as much as they needed to (Figure III.2). The relatively high 
percentage of supervisors who agreed with this statement suggests that, although reducing span 
of control is important for clearing the way for supervisors to spend time in schools, it is not a 
panacea. In some PSI districts, demands and expectations for supervisors’ time from other 
central office departments persisted. Despite their reduced caseloads, supervisors were still 
required to attend to these demands, such as addressing parent complaints or attending to 
administrative requests from the central office. 


Figure I11.2. Principal supervisors’ perceptions of their capacity to support 
principals in PSI and other urban districts 
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Figure reads: Thirteen percent of principal supervisors in PSI districts with a span of control of fewer than 13 
principals agreed or strongly agreed that they supervise too many principals to provide them with enough support. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: Survey question read, “Based on your experiences in the current 2017-2018 school year, how strongly do 
you agree or disagree with the following statements?” Supervisors rated their agreement on a 5-point 
Likert-type scale. Differences between supervisors in PSI and other urban districts are not statistically 
significant at the 5 percent level for either item. Differences between lower and higher span of control 
groups are statistically significant at the 5 percent level for both items. 


To further help supervisors focus on developing principals’ instructional leadership skills, 
and to reduce administrative work, the PSI encouraged districts to reduce principal supervisors’ 
responsibilities overseeing non-instructional personnel. During the PSI, as supervisors focused 
more intensely on supporting schools and principals, the percentage of supervisors supervising 
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non-instructional personnel declined. In both PSI and other urban districts, however, most 
supervisors oversaw some personnel who were not principals. An equal proportion—about 
half—of principal supervisors in PSI and other urban districts reported that they had at least one 
direct report who was not a principal, such as a secretary or principal coach. Among supervisors 
with any direct reports, those in other urban districts had twice as many direct reports, an average 
of six, as supervisors in PSI districts, an average of three (Figure III.3). 


Figure I11.3. Average number of direct reports overseen by principal supervisors 
in PSI and other urban districts, among supervisors who had at least one direct 
report 
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Figure reads: Supervisors who had direct reports in PSI districts oversaw an average of three direct reports. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: Survey question read, “How many personnel directly reported to you?” Only respondents who answered 
“Yes” to the question “Not including principals, during the 2017—2018 school year, did other school or 
district personnel report directly to you?” answered the question (supervisors in other urban districts, n = 
139; supervisors in PSI districts, n = 24). Difference between supervisors in PSI and other urban districts is 
statistically significant at the 5 percent level. 


Many of the PSI districts implemented innovative models to support supervisors and 
schools. Some PSI districts replaced traditional administrative models of support with cross- 
functional teams, often called central office support teams (Goldring et al. 2018). In the support 
team model, supervisors worked with designated personnel from other central departments, such 
as instructional coaches (Figure III.4). Although these personnel reported to their own 
department heads, supervisors were able to draw on their support and sometimes deploy them to 
schools as they saw fit, rather than spend time supervising these staff positions administratively. 
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Figure II1.4. District support team structure 
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Source: PSI district interviews with central office personnel and principal supervisors, 2015 and 2016. Adapted from 
Goldring et al. 2018. 


Approach to assigning principals to supervisors and networks 


In addition to reducing the supervisors’ span of control, PSI districts also redesigned how 
they grouped or assigned principals and supervisors to networks to maximize the effectiveness of 
supervisor support and principals’ learning opportunities with other principals in their network. 
The PSI districts assigned supervisors to networks based on grade level, geographic location, 
feeder patterns, or school performance. Each approach presented unique benefits and trade-offs 
for supervisors and principals (Goldring et al. 2018). Most PSI districts relied on more than one 
of these criteria for grouping principals. 


Approaches to assigning principals to supervisors and networks differed across the PSI and 
other urban districts (Figure III.5). Supervisors in PSI districts were more likely to oversee 
groups of principals who were organized by grade level (such as middle school principals), 
whereas those in other urban districts were more likely to oversee groups of principals organized 
by attendance feeder pattern (groups of elementary, middle, and high schools serving the same 
students over time). 


These differences may reflect the large amount of thought that the PSI districts put into 
redesigning principal networks. Before the PSI, the prevailing method of supervisor assignment 
in most districts was by geographic area. As PSI districts grappled with increasing the 
instructional focus of the supervisor role, assignment based on academic features of schools— 
such as grade level, performance, or theme—allowed districts to match supervisors to schools 
based on their instructional expertise. To reduce supervisors’ travel time, however, most PSI 
districts also attempted to take school location into account in the groupings. 
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Figure 111.5. Criteria for principal assignment to supervisors across PSI and 
other urban districts 
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Figure reads: Seventy-three percent of supervisors in PSI districts indicated that grade level was a criterion by which 
their principals were assigned to them. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: Survey question read, “This year, by what criteria are the majority of your principals assigned to you?” 
Respondents could choose multiple categories. Differences between supervisors in PSI and other urban 
districts are statistically significant for “Grade level” and “Feeder patterns” only. 


B. Principal supervisor training and professional development 


Before the PSI, supervisors in PSI districts received minimal training on improving their 
capacity to support and develop principals as instructional leaders. Instead, they typically 
received generic professional development alongside principals and other central office staff. 


In contrast, during the PSI, all six districts implemented dedicated training programs to 
develop supervisors’ capacity to support and coach principals. Technical assistance providers 
played key roles in planning and facilitating the training. Supervisors especially valued job- 
embedded training approaches, such as one-on-one coaching and in-school peer observations 
with other supervisors. In the final year of the PSI, training in some districts declined in 
frequency and quality as these districts transitioned from more expensive external training 
providers to in-house trainings (Goldring et al. 2020). When not protected as a priority, these in- 
house trainings risked being overtaken by district administrative items and information sharing. 
However, some of the PSI districts committed to making dedicated, in-house principal supervisor 
training a priority, which ensured the continued success and growth of their principal 
supervisors’ knowledge and expertise. 
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Access to role-specific training 


Consistent with the PSI’s emphasis on providing specific, targeted training to principal 
supervisors, supervisors in PSI districts were significantly more likely than those in other urban 
districts to report participating in professional development related to their role as a principal 
supervisor (Figure III.6). Eighty percent of supervisors in PSI districts reported that they 
participated in training specific to their role in the 2017—2018 school year, compared with only 
62 percent of supervisors in other urban districts. 


Figure I11.6. Principal supervisors’ participation in role-specific training in PSI 
and other urban districts 
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Figure reads: Eighty percent of principal supervisors in PSI districts participated in training specific to their role. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “During the 2017-2018 school year and the summer that preceded it, did you 
participate in district-sponsored training or professional development pertaining to your specific role as a 
principal supervisor?” Differences between supervisors in PSI and other urban districts are statistically 
significant at the 5 percent level. 


Perceptions of training quality 


Among supervisors who reported receiving role-specific training, those in PSI districts rated 
the quality of this training significantly more highly than did supervisors in other urban districts 
according to a scale measuring training quality (Figure II.7). The scale asked supervisors to rate 
their level of agreement with 20 statements pertaining to the quality of the role-specific 
professional development or training they received during the previous year and the summer 
before. For example, they were asked about their agreement with the statements that the 
professional development “specifically enhanced my capacity to develop principals’ instructional 
leadership,” “helped build a learning community with my fellow supervisors,” and “provided 
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opportunities to share specific practices with other principal supervisors in my district.” Overall, 
principal supervisors in both PSI and other urban districts agreed that their training was high 
quality. In general, however, supervisors in PSI districts were more likely than those in other 
urban districts to agree with the survey statements about the quality of their professional training. 


Figure III.7. Principal supervisors’ perceptions of the quality of their 
professional development and training in PSI and other urban districts 
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Figure reads: Supervisors in PSI districts rated their training quality an average of 3.7 on a 5-point scale. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Thinking about the district-sponsored training or professional development for 
principal supervisors you attended during the 2017-2018 school year and the summer before, to what 
extent do you agree or disagree with each of the following statements? ‘The professional development or 
training | attended....”” The scale was made up of 20 items; see the appendix for information on scale 
construction. Supervisors rated their agreement with each item on a 5-point Likert-type scale. Differences 
between supervisors in PSI and other urban districts are statistically significant at the 5 percent level. 


Training emphasis 


Across PSI and other urban districts, principal supervisors’ training emphasized similar 
skills and practices. Surveys asked supervisors to indicate the level of emphasis of their 
professional development across a range of topics. Across both PSI and other urban districts, 
supervisors indicated that the skills most emphasized in their trainings were observing 
classrooms to identify instructional quality, improving student growth and achievement, using 
student performance data to improve classroom instruction, skills for effectively coaching 
principals, and providing actionable and specific feedback to principals (Figure III.8). 
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Figure II1.8. Five most common areas of emphasis in principal supervisor 
trainings in PSI and other urban districts 
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Figure reads: Forty-one percent of supervisors in PSI districts indicated that their training placed a great deal of 
emphasis on observing classrooms to identify instructional quality. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Thinking about the district-sponsored training or professional development for 
principal supervisors you attended during the 2017-2018 school year and the summer before, how much 
emphasis was placed on the following areas?” The set comprised 23 items. Supervisors indicated their 
response as “no emphasis,” “some emphasis,” or “a great deal of emphasis.” Figure shows the percentage 
of supervisors who marked “a great deal of emphasis” in each category. Differences between supervisors in 
PSI and other urban districts are not statistically significant at the 5 percent level. 


Three areas—working effectively one-on-one with principals, helping principals provide 
actionable and specific feedback to teachers, and using resources in their work—were among the 
skills most emphasized in trainings in other urban districts but not in PSI districts (not shown). 
These differences in emphasized skills may reflect the evolution of the training that supervisors 
in PSI districts received during the initiative. In the earlier years of the initiative, training in the 
PSI districts focused on providing effective one-on-one support and specific feedback to 
principals. In later years (including the year of the survey), the focus of the trainings shifted 
toward adapting and differentiating support based on principals’ needs and away from 
developing specific skills and practices (Goldring et al. 2020). 


Despite these differences, training for supervisors in both PSI and other urban districts 
clearly emphasized skills for improving instruction and instructional leadership. Only 9 percent 
of supervisors in PSI districts and 5 percent of those in other urban districts indicated that their 
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training placed a great deal of emphasis on working with principals on operations, human 
resources, or budget management. 


C. Central office support for the principal supervisor role 


The PSI emphasized the importance of the central office in supporting and sustaining the 
new principal supervisor role. Central offices are responsible for (1) selecting and training future 
supervisors (including mentoring and preparing aspiring supervisors), and (2) creating staff roles 
and structures to support supervisors’ work so those supervisors can meet the needs of the 
principals and their schools. In the PSI districts, the central offices that changed their structures 
and orientation toward supporting schools were better able to support principal supervisors 
(Goldring et al. 2020). 


Selecting and training future principal supervisors 


Administrative turnover and vacancies can present challenges to the central office’s attempts 
to support schools. To ensure that high quality principal supervisor candidates are available for 
position openings, some districts operate apprenticeship programs to recruit and train promising 
aspiring principal supervisors. Other districts, particularly those with little principal supervisor 
turnover, may prefer more informal succession planning. In addition, some districts may provide 
formal mentoring to new principal supervisors to help them transition smoothly into their new 
role. 


The PSI districts took varying approaches to succession planning. Three of the six PSI 
districts implemented supervisor apprenticeship programs. The other three districts monitored 
leadership in the district to identify promising future supervisor candidates (Goldring et al. 
2018). The districts that chose to create apprenticeship programs varied in their approach, but 
common features of each program were a rigorous selection process, ongoing and job-embedded 
opportunities for trainees to experience the supervisor role, and coaching and feedback from 
mentors and experts. The districts that chose to not formally implement an apprenticeship 
program often relied on systems for tracking and identifying promising candidates. 


Supervisors in PSI districts were more likely than those in other urban districts to report that 
their district had an aspiring supervisor program, as well as mentorship programs for new 
supervisors (Figure III.9). Fifty-nine percent of supervisors in PSI districts reported that their 
district currently had a program to identify and prepare aspiring principal supervisors, whereas 
only 19 percent of supervisors in other urban districts reported that their district currently had an 
aspiring supervisor program. In fact, 20 percent of supervisors in the PSI districts were hired into 
their position through their participation in an aspiring supervisor program, compared with just 5 
percent of supervisors in other urban districts (not shown). PSI districts also were more likely to 
offer mentoring or induction programs for new supervisors. Forty-nine percent of supervisors in 
PSI districts reported that their district had a mentoring or induction program for new 
supervisors, compared with 22 percent of supervisors in other urban districts. 
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Figure III.9. Aspiring supervisor and supervisor induction programs in PSI and 
other urban districts 
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Figure reads: Fifty-nine percent of supervisors in PSI districts reported that their district has a program for aspiring 
principal supervisors. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey questions read, “Does your district have a program in place to identify and prepare aspiring 
principal supervisors?” and “Does your district have a mentoring or induction program for new principal 
supervisors?” Supervisors indicated “yes” or “no.” Differences between supervisors in PSI and other urban 
districts are statistically significant at the 5 percent level for both items. 


Three of the six PSI districts drove the large differences between PSI and other urban 
districts in the reported availability of aspiring principal supervisor programs and induction 
programs. Broward, Cleveland, and Long Beach created formal aspiring supervisor programs 
that focused on recruiting promising principal supervisor candidates from among the ranks of top 
principals and cultivating their skills through internship and study (Goldring et al. 2018). These 
districts also created mentoring programs available for newly hired supervisors, although these 
programs ranged in formality. Aspiring and mentoring/induction programs for supervisors 
required much district investment and coordination to find resources and develop supports for 
supervisor candidates. Some PSI districts, including Des Moines and Minneapolis, found that 
aspiring supervisor programs were not well aligned with their organizational models. Instead, 
they relied on talent management tools (such as leader tracking systems, or data systems to track 
information about candidates for future leadership positions and current available positions) to 
help them identify promising supervisor candidates. 
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Principal supervisors’ perceptions of central office support for principals and supervisors 


Early in the initiative, leaders in the PSI districts realized that, to change the principal 
supervisor role, they would also need to restructure tasks and responsibilities in the central office 
and change its culture to better support schools (Goldring et al. 2018). The PSI districts created 
structures, such as central office support teams made up of personnel from other departments, 
that helped supervisors coordinate school support with other central office personnel. In addition, 
most of the PSI districts changed their central office schedules to free up supervisors’ time so 
meetings were not held during school hours. 


Principal supervisors in PSI and other urban districts had similar perceptions of the support 
they received from the central office. Overall, fewer than half of supervisors in PSI and other 
urban districts agreed that the central office understood their work, facilitated their work with 
principals, was organized to support principals, or scheduled meetings to maximize supervisors’ 
time in schools (Figure III.10). In addition, only about one-third of supervisors in both PSI and 
other urban districts agreed that the central office was organized to support principals. Although 
supervisors’ responses to these items indicate that there was room for improvement in the central 
office’s support for principal supervisors, most supervisors in PSI and other urban districts 
agreed that improving teaching and learning in schools was a key focus of the central office’s 
work. 


Supervisors’ relatively negative perceptions of the support they received from the central 
office in PSI districts results speak to the challenges central office leaders in PSI districts faced 
in changing the culture of the central office. District leaders in multiple PSI districts spoke of 
changing central office structures and changing central office cultures as two separate 
challenges. Although leaders were able to make headway in changing central office structures, 
such as altering meeting times to allow supervisors more time in schools and creating support 
teams and liaison structures to support supervisors’ work, they said it was more difficult to 
change the central office culture to one that supported teaching and learning (Goldring et al. 
2018). 


Principal supervisors in PSI districts were more likely than supervisors in other urban 
districts to agree with several statements about the quality and effectiveness of the support they 
received from the central office (Figure III.10). For example, supervisors in PSI districts were 
more likely than those in other urban districts to agree that they were involved in the deployment 
of instructional support staff to the schools they supervised (71 versus 41 percent). A slightly 
larger proportion of supervisors in PSI districts agreed that central office meetings were 
scheduled so that they could maximize their time in schools (41 versus 35 percent) and that these 
meetings were useful to their practice (43 versus 34 percent), although these differences were not 
statistically significant. 


A lower proportion of supervisors in PSI districts than in other urban districts agreed that 
improving teaching and learning was a key focus of the central office’s work (59 versus 71 
percent) and that other departments understood their work (27 versus 36 percent), although these 
differences were not statistically significant. The relatively low rates of agreement with these 
statements among supervisors in PSI districts highlight some of the challenges the PSI districts 
faced in changing central office culture to align with the revised principal supervisor role. As the 
PSI districts shifted the principal supervisor role away from managerial tasks, some supervisors 
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noted that other departments in the central office remained mired in regulation and compliance. 
Through their unique training, interactions with other supervisors in the PSI, and their focus on 
their new role, supervisors in PSI districts had specific ideas of how the central office could 
support their work and be more responsive to their principals. These supervisors occasionally 
expressed frustration that their new role was not well understood by other central office 
departments and that these departments were slow in learning to support them in their work of 
helping principals improve teaching and learning. Supervisors in other urban districts might not 
have felt this disconnect, because they might not have had to navigate such significant changes to 
their role or been part of deliberations about central office redesign to support their roles. 


Improving central office culture was slow but possible. Supervisors’ perceptions of the 
central office in PSI district over the course of the initiative indicated incremental but positive 
and statistically significant changes (Goldring et al. 2020). 


Figure 111.10. Principal supervisors’ perceptions of the central office in PSI and 
other urban districts 
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Figure reads: Twenty-seven percent of supervisors in PSI districts agreed or strongly agreed that departments in the 
central office understand their work. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences with this district's central office in the current 201 7— 
2018 school year, how strongly do you agree or disagree with the following statements?” Supervisors rated 
their agreement on a 5-point Likert-type scale. Difference between supervisors in PSI and other urban 
districts is statistically significant at the 5 percent level for the item “I am involved in the deployment of 
instructional support staff to the schools | supervise.” 
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In both PSI and other urban districts, large proportions of supervisors agreed with several 
statements regarding barriers from the central office that they encountered with their work with 
principals (Figure [I.11). For example, most supervisors agreed that principals lost time 
focusing on teaching and learning because of requests from the central office. These findings 
again underscore the difficulties of reorienting the central office system to support principals and 
schools, even as the PSI districts were able to successfully redefine the role of principal 
supervisors themselves. 


However, a smaller proportion of supervisors in PSI districts reported that their principals 
sought help from them because they did not know who to contact in the central office (49 versus 
61 percent of supervisors in other urban districts) or that instructional staff were deployed to the 
central office without their knowledge (43 versus 57 percent of supervisors in other urban 
districts), although these differences were not statistically significant. Our interviews with 
principal supervisors in PSI districts suggest that they have become more aware of, and involved 
in, central office processes for school support, likely due to the introduction of innovative 
collaborative structures such as central office support teams and department liaison systems. 


Figure 111.11. Principal supervisors’ perceptions of barriers from central office in 
PSI and other urban districts 
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Figure reads: Thirty-one percent of supervisors in PSI districts agreed or strongly agreed that the way the central 
office is organized interferes with their ability to work with other principal supervisors. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences with this district's central office in the current 2017— 
2018 school year, how strongly do you agree or disagree with the following statements?” Supervisors rated 
their agreement on a 5-point Likert-type scale. Differences between supervisors in PSI and other urban 
districts are not statistically significant at the 5 percent level. 
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Evaluation of principal supervisors 


Supervisors themselves may receive evaluations and feedback on their performance from 
their superiors in the central office. However, few districts have implemented evaluations that 
are specific to principal supervisors. As the supervisor role continues to evolve away from 
administration and compliance, general evaluations may no longer provide supervisors with the 
specific, targeted feedback they require to improve their performance. 


None of the PSI districts had formal supervisor-specific evaluation systems at the beginning 
of the PSI. When districts evaluated supervisors at all, they used a generic central office rubric 
that was ill-equipped to capture supervisors’ effectiveness in the changing role. Recognizing the 
importance of providing principal supervisors with ongoing feedback, four PSI districts invested 
in formalizing evaluation systems or tools that assessed supervisors according to the expectations 
for their revised role. For example, Cleveland moved away from a generic central office 
evaluation rubric to a detailed assessment that included monthly meetings with the department 
chief, coaching observations, and structured write-ups about each supervisors’ performance, as 
well as improvement plans for supervisors who needed them. The other two PSI districts 
provided informal coaching and feedback to principal supervisors in regular meetings with their 
supervisor at the central office. Several PSI district leaders said they would continue to develop 
and refine their principal supervisor evaluation systems after the PSI ended. 


Overall, supervisors in PSI districts expressed more positive views of their supervisor 
evaluation systems than did supervisors in other urban districts (Figure HI.12). Supervisors in 
PSI districts were more likely than those in other urban districts to agree that their district’s 
principal supervisor evaluation system was aligned with their work (55 versus 39 percent) and 
provided actionable feedback (65 versus 41 percent). They also were more likely to report that 
their district’s principal supervisor evaluation system incorporated principal feedback (57 versus 
21 percent). However, in both PSI and other urban districts, about a quarter of supervisors (29 
and 23 percent) agreed that the principal supervisor evaluation system in their district was very 
general. 
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Figure 111.12. Principal supervisors’ perceptions of the quality of their district’s 
principal supervisor evaluation system in PSI and other urban districts 
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Figure reads: Fifty-five percent of supervisors in PSI districts agreed or strongly agreed with the statement “The 
principal supervisor evaluation system in this district aligns with my role and the work | do.” 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences in the current 2017-2018 school year, how strongly 
do you agree or disagree with the following statements about the principal supervisor evaluation system in 
this district?” Supervisors rated their agreement on a 5-point Likert-type scale. Differences between 
supervisors in PSI and other urban districts are statistically significant at the 5 percent level for “The 
principal supervisor evaluation system in this district aligns with my role and the work | do,” “I receive 
actionable and useful feedback from my supervisors’ evaluations of my performance,” and “Principals 
provide formal input into my evaluation.” 


Supervisors in PSI districts were more likely than those in other urban districts to report that 
their principal supervisor evaluation system held them accountable for various outcomes (Figure 
III.13). For example, 65 percent of supervisors in PSI districts reported that their evaluation 
system held them accountable for improving student achievement, compared with 51 percent of 
those in other urban districts. However, none of these differences was statistically significant. 
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Figure 111.13. Principal supervisors’ perceptions that their district’s evaluation 
system held them accountable in PSI and other urban districts 
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Figure reads: Fifty-three percent of supervisors in PSI districts agreed or strongly agreed with the statement “The 
principal supervisor evaluation system in this district holds me accountable for improving achievement outcomes of 
special education students.” 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences in the current 2017-2018 school year, how strongly 
do you agree or disagree with the following statements about the principal supervisor evaluation system in 
this district?” Supervisors rated their agreement on a 5-point Likert-type scale. Differences between 
supervisors in PSI and other urban districts are not statistically significant at the 5 percent level. 


D. Summary 


Early in the PSI, districts made wide-ranging structural changes to foster the redesign of the 
supervisor role, and they continued to refine these structures throughout the initiative. By the end 
of the PSI, when supervisors completed the survey, clear differences were apparent between PSI 
and other urban districts. Principal supervisors in PSI districts had lower average spans of 
control, fewer direct reports but greater agency in deploying central office staff, more and better 
role-specific training, and access to the role through apprenticeship and mentorship programs. 


Because these changes are resource- and time-intensive, they require planning and 
coordination among multiple district stakeholders that would be unlikely to occur without a 
strong vision for the new supervisor role. Even with the clear vision of the PSI, district leaders 
sometimes met with resistance as they made these changes, especially when the new principal 
supervisor role was not yet well understood across the central office. Many PSI district leaders 
reported that changing systems and mindsets in the central office was one of the most 
challenging aspects of the initiative. These challenges were reflected in principal supervisors’ 
perceptions of the support they received from the central office even at the end of the initiative. 
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IV. PRINCIPAL SUPERVISORS’ ROLES AND PRACTICES 


Changing the principal supervisor role to better support principals requires altering the day- 
to-day practices of principal supervisors. Before the PSI, districts gave supervisors broad 
discretion in their work with principals, outside of formal evaluation and compliance 
responsibilities. Supervisors in PSI districts described their role before the initiative as reactive, 
requiring them to respond to every need that principals voiced. In contrast, the PSI focused the 
principal supervisor role on providing direct supports to principals and developing and coaching 
them. PSI districts also strived to align principal supervisors’ evaluations with their work with 
principals. 


A. Principal supervisors’ time use 


Because principal supervisors typically have many roles, they must spend their time on a 
variety of activities. These include principal coaching and evaluation, district administration, and 
leading principal professional learning communities. In a previous report by the Council of the 
Great City Schools, 80 percent of principal supervisors noted that they were responsible for 
addressing district administrative issues and 62 percent noted that they were responsible for 
addressing district compliance issues in 2012 (Casserly et al. 2013). 


The PSI attempted to further focus each principal supervisor’s time on direct interactions 
with principals and their leadership teams. The PSI districts used several strategies to free up 
supervisors’ time to spend in schools with principals. For example, some districts scheduled 
central office meetings after school hours so that they would not conflict with supervisors’ 
school visits, mandated a minimum number of visits supervisors should make to each principal, 
and reassigned managerial and non-instructional responsibilities from supervisors to other central 
office staff. Several districts also developed systems to help supervisors differentiate their time 
based on principals’ needs, especially in the later years of the initiative (Goldring et al. 2020). 


Despite the lower span of control in PSI districts, supervisors in both PSI and other urban 
districts had similar patterns of time use and focused much of their efforts on instructional 
leadership work with principals. This pattern is consistent with the national standards for 
principal supervisors (Council of Chief State School Officers 2015). 


Overall time use 


Principal supervisors in both PSI and other urban districts spent their time in a similar 
manner (Figure IV.1). In both sets of districts, supervisors spent about half their time visiting 
schools and about 15 percent of their time in network and group meetings with principals, with a 
total of about 65 percent of their time working and interacting with principals. Differences 
between time allocation in PSI and other urban districts were generally small. 


25 


LEADING THE CHANGE: A COMPARISON OF PSI AND OTHER URBAN DISTRICTS VANDERBILT UNIVERSITY / MATHEMATICA 


Figure IV.1. Principal supervisors’ time use in PSI and other urban districts 
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Figure reads: Principal supervisors in PSI districts spent 47 percent of their time visiting schools. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Over the past three months, what percentage of time did you spend on each of 
the following activities in a typical week, excluding travel time?” Differences between supervisors in PSI and 
other urban districts are statistically significant at the 5 percent level for “In meetings with other PS only” 
and “Other.” Totals across PSI and other urban districts may not sum to 100 due to rounding. 


Number of visits to schools 


Although supervisors in both PSI and other urban districts spent about half of their time, on 
average, visiting schools, some schools received more frequent visits than others. For example, 
in the first three years of the PSI, some principals reported receiving no visits from their 
supervisor during a three-month period, whereas others reported receiving as many as 20 visits 
(Goldring et al. 2018). In later years, these patterns persisted and could be linked to supervisors’ 
span of control: principals whose supervisors had spans of control of 11 principals or fewer 
received an average of about five visits per semester, but principals whose supervisors oversaw 
15 principals or more received only about three visits in the same time period (Goldring et al. 
2020). This suggests that span of control influenced how often supervisors in PSI districts were 
able to visit their schools. 


To further understand how frequently supervisors visited their schools, the survey asked 
them to report how many times they visited their least- and most-visited schools in a three-month 
period. Compared with supervisors in other urban districts, supervisors in PSI districts reported 
making slightly more visits to their least- and most-visited schools (Figure IV.2), although these 
differences were not statistically significant. On average, supervisors in PSI districts made about 
one more visit to their least-visited schools and about three more visits to their most-visited 
schools in a three-month span, compared with those in other urban districts (not statistically 
significant). These results are consistent with the research finding that supervisors with lower 
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spans of control make more visits to each principal, especially those that need the most support 
(Goldring et al. 2018 and 2020). Many PSI districts used “tiering,” a process in which 
supervisors categorized schools into three tiers of support based on needs and then made more 
visits to the schools that needed support the most, while still maintaining a minimum number of 
visits to other schools. 


Figure IV.2. Number of principal supervisors’ visits to least- and most-visited 
schools in PSI and other urban districts over past three months 
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Figure reads: Supervisors in PSI districts reported making an average of five visits to their least-visited school in the 

past three months. 

Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: Survey question read, “Please think about the school you visited LEAST (MOST) frequently over the past 
three months among the schools that you supervise. How many times did you visit this school?” Differences 
between supervisors in PSI and other urban districts are not statistically significant at the 5 percent level. 


Instructional leadership focus with principals 


In a typical week, principal supervisors in both PSI and other urban districts spent about half 
their time with principals focusing on instructional leadership and the other half focusing on 
other topics (Figure IV.3). Differences in the time spent on each focus area between supervisors 
in PSI and other urban districts were small and not statistically significant. 
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Figure IV.3. Focus of principal supervisors’ work with principals in PSI and other 
urban districts 
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Figure reads: Principal supervisors in PSI districts spent 52 percent of their time with principals focusing on 
instructional leadership. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Over the past three months, what percentage of time did you spend working 
with principals on each of the following in a typical week, excluding travel time?” Differences between 
supervisors in PSI and other urban districts are not statistically significant at the 5 percent level. 


B. Principal supervisors’ practices to support principals 


The structures PSI districts implemented to support principal supervisors were intended to 
foster supervisors’ ability to develop, engage in, and sustain the day-to-day practices necessary 
to improve principals’ leadership. Principal supervisors use a variety of practices with their 
principals. Before the PSI, the PSI districts mostly left the work of principal supervision to the 
discretion of each supervisor (except for compliance responsibilities). During the PSI, these 
districts worked to introduce and strengthen supervisors’ practices to develop and support 
principals’ instructional leadership and promote consistency in these practices. These practices 
included classroom walkthroughs, feedback, coaching, facilitating principal learning 
communities, and principal evaluation. 


Supervisors’ practices to support principals in their visits to schools were similar across PSI 
and other urban districts (Figure IV.4). Differences between supervisors in PSI and other urban 
districts were minimal for other practices such as modeling effective teaching practices, 
preparing a specific agenda in advance, and communicating the goals of their work with 
principals. Only one practice, modeling effective feedback and coaching with principals, was 
significantly different between PSI and other urban districts (59 versus 76 percent). 
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Figure IV.4. Principal supervisors’ practices with principals during school visits 
in PSI and other urban districts 
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Figure reads: Thirty-one percent of supervisors in PSI districts reported that they usually or always modeled effective 
teaching practices when visiting with principals at school. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Over the past three months, when you visited a principal at his/her school, how 
often did you do each of the following?” Supervisors rated their frequency of practices on a 5-point 
frequency scale. Difference between supervisors in PSI and other urban districts is statistically significant at 
the 5 percent level for “Modeled effective feedback and coaching.” 


Supervisors in PSI and other urban districts also engaged in similar practices with principals 
outside of school visits (Figure IV.5). Supervisors in PSI districts were significantly less likely 
than those in other urban districts to report that they usually or always provided principals with 
actionable feedback (76 versus 94 percent). 
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Figure IV.5. Principal supervisors’ instructional leadership practices with 
principals (not specific to school visits) in PSI and other urban districts 
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Figure reads: Forty-nine percent of supervisors in PSI districts reported that they usually or always used a specific 
protocol when discussing data with principals. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Thinking about all the time you spent working with principals over the past three 
months, how often would you say each of the following were true?” Supervisors rated their frequency of 
practices on a 5-point frequency scale. Difference between supervisors in PSI and other urban districts is 
statistically significant at the 5 percent level for “Provided principals with actionable feedback.” 


In contrast, supervisors in PSI districts worked less often with principals on hiring and 
operations than supervisors in other urban districts (Figure IV.6). Twenty-two percent of 
supervisors in other urban districts reported that they usually or always supported principals with 
hiring teachers or other school staff, compared with 10 percent of those in PSI districts. Nearly 
half (45 percent) of supervisors in other urban districts also indicated that they usually or always 
helped principals with facilities or other operational issues, whereas only 20 percent of 
supervisors in PSI districts indicated this. Supervisors in PSI districts continued to support 
principals, especially new ones, with annual budgeting (Goldring et al. 2020). 
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Figure IV.6. Principal supervisors’ work with principals on hiring and operational 
issues in PSI and other urban districts 
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Figure reads: Ten percent of supervisors in PSI districts reported that they usually or always supported principals with 
hiring teachers or other school staff. 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Thinking about all the time you spent working with principals over the past three 
months, how often would you say each of the following was true?” Supervisors rated their frequency of 
practices on a 5-point frequency scale. Differences between supervisors in PSI and other urban districts are 
statistically significant at the 5 percent level for “I supported principals with hiring teachers or other school 
staff’ and “I helped principals with facilities or other operational issues.” 


Given the intensive training and support in instructional leadership practices that supervisors 
in PSI districts received as a part of the initiative, the lack of consistent differences between PSI 
and other urban principal supervisors in their use of practices related to instructional leadership 
may seem counterintuitive. There are a few possible explanations for these results. 


First, there has been a nationwide emphasis (beyond the PSI districts) on refocusing 
principal supervision. For example, a recent report from the Council of the Great City Schools 
showed that, between 2012 and 2018, principal supervisors’ average span of control in their 
member districts had decreased from 24 to 16 (Cochran et al. 2020). Other urban districts may 
have given their supervisors guidance for working with principals that was comparable to that 
given in the PSI districts. The Council of the Great City Schools reported that, in 2018, many 
more supervisors were engaging in instructional leadership practices with principals (such as 
visiting classrooms and discussing feedback from classroom walkthroughs) than in 2012. 
Furthermore, the nationwide diffusion of new standards and expectations for principal supervisor 
work, such as those presented in the 2015 Model Principal Supervisor Standards (Council of 
Chief State School Officers 2015), likely influenced other urban districts to make changes to the 
principal supervisor role. 
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Second, supervisors in PSI districts may have underestimated their engagement with 
instructional leadership practices, given the extensive training they received on these practices. 
Research has documented the widespread phenomenon that people often overestimate their 
performance in areas in which they have not received training or do not have a nuanced 
understanding of a practice, whereas people who have received instruction and training in an 
area tend to underestimate their performance (Kruger and Dunning 1999). Many of the practices 
supervisors reported on revolve around complex concepts such as modeling teacher feedback 
conversations, coaching principals, monitoring their growth and change, and providing them 
with actionable feedback. Supervisors in PSI districts received extensive training in these areas 
and often worked in teams to implement structures and protocols designed to guide these 
practices and observed each other in job-embedded sessions in schools (Goldring et al. 2018). 
For example, supervisors in Des Moines Public Schools spent the first two years of the initiative 
in intensive trainings in which they studied the district’s teacher evaluation standards in detail 
and worked together to form a consistent understanding of how to support principals in 
observing, rating, and giving feedback to teachers. In Long Beach, supervisors participated in 
special training lab days on which they observed and critiqued each other engaging in 
instructional leadership coaching with principals. Cleveland and Minneapolis hired coaches to 
work directly with supervisors to help them reflect and address problems of practice. As a result 
of unique and dedicated training, supervisors in PSI districts may have deepened their knowledge 
of how instructional leadership is defined and may have consequently formed higher 
expectations and a more specific understanding of what these practices entail. In contrast, 
supervisors who were not specifically trained in these areas may have inaccurately believed they 
were engaging in these high-leverage practices frequently and may have misunderstood what 
these practices entail. 


C. Principal evaluation 


Nearly all principal supervisors in the PSI districts evaluated principals. Prior surveys in the 
PSI districts showed that 100 percent of supervisors in PSI districts evaluated principals by the 
third year of the initiative (Goldring et al. 2018). Because intensive principal coaching and 
feedback were hallmarks of the new principal supervisor role, some districts that implemented 
the PSI invested in developing and refining their principal evaluation systems to align with 
principal supervisors’ changing role, focusing on providing more useful feedback to principals 
(Goldring et al. 2018). The goal was to use principal evaluation systems to align supervisors’ 
work with ongoing formative feedback for principals. 


A larger proportion of supervisors in PSI districts than in other urban districts agreed that the 
principal evaluation system was aligned with the work supervisors and principals did together 
(76 versus 63 percent; Figure IV.7), although this difference was not statistically significant. 
Furthermore, a larger proportion of supervisors in PSI districts than in other urban districts 
reported that the principal evaluation system provided actionable feedback for principals (67 
versus 57 percent), although this difference was also not statistically significant. 
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Figure IV.7. Principal supervisors’ views of the alignment of their districts’ 
principal evaluation systems with their work in PSI and other urban districts 
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Figure reads: Seventy-six percent of supervisors in PSI districts agreed with the statement “The principal evaluation 
system in this district aligns with the ongoing work | do with my principals.” 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences in the current 2017-2018 school year, how strongly 
do you agree or disagree with the following statements about the principal evaluation system in this 
district?” Supervisors rated their agreement on a 5-point Likert-type scale. Differences between supervisors 
in PSI and other urban districts are not statistically significant at the 5 percent level. 


Compared with supervisors in PSI districts, those in other urban districts were more likely to 
agree with negative statements about their principal evaluation system (Figure IV.8). Forty 
percent of supervisors in other urban districts agreed that it was unclear how principal evaluation 
data were used in the district, compared with only 18 percent in PSI districts. Similarly, 38 
percent of supervisors in other urban districts agreed that the principal evaluation system was too 
cumbersome, compared with only 27 percent in PSI districts, but this difference was not 
statistically significant. 
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Figure IV.8. Principal supervisors’ ratings of their district’s principal evaluation 
system in PSI and other urban districts 
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Figure reads: Eighteen percent of supervisors in PSI districts agreed with the statement “It is unclear how principal 
evaluation data are used in this district.” 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences in the current 2017-2018 school year, how strongly 
do you agree or disagree with the following statements about the principal evaluation system in this 
district?” Supervisors rated their agreement on a 5-point Likert-type scale. Difference between supervisors 
in PSI and other urban districts is statistically significant at the 5 percent level for “It is unclear how principal 
evaluation data are used in this district.” 


Improving principal evaluation was not an explicit focus of the PSI, but districts used their 
evaluation systems to help principal supervisors use benchmarks and data to be more effective 
coaches and evaluators. During these trainings, supervisors became more familiar with 
identifying principal effectiveness and using district evaluation rubrics to rate and provide 
actionable feedback to principals. Supervisors developed expert knowledge in the competencies 
principals needed to lead schools effectively and adapted their own work accordingly. They 
attended sessions to learn how to calibrate their ratings and justify evidence in those ratings. 
Some districts, such as Des Moines, also worked to improve their principal evaluation systems 
themselves to ensure that they aligned with district expectations for principals’ instructional 
leadership, as well as the coaching and reflection work that principals were expected to do with 
their supervisors. 


Finally, perhaps as a result of the emphasis on evaluation as an integral part of the 
supervisor role, there was a strong link between principal evaluation and accountability in PSI 
districts. Supervisors in PSI districts were significantly more likely than those in other urban 
districts to report that their principal evaluation systems held principals accountable for several 
outcomes (Figure IV.9). For example, 67 percent of supervisors in PSI districts agreed or 
strongly agreed that their principal evaluation system held principals accountable for improving 
achievement outcomes of students with special needs, whereas only 29 percent of supervisors in 
other urban districts agreed with this item. 
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Figure IV.9. Principal supervisors’ perceptions of the district’s principal 
evaluation system for principal accountability in PSI and other urban districts 
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Figure reads: Thirty-nine percent of supervisors in PSI districts agreed with the statement, “The principal supervisor 
evaluation system in this district effectively holds principals accountable for improving student attendance.” 


Source: Wallace Foundation National Survey of Principal Supervisors, 2018 (supervisors in PSI districts, n = 50; 
supervisors in other urban districts, n = 293). 


Note: The survey question read, “Based on your experiences in the current 2017—2018 school year, how strongly 
do you agree or disagree with the following statements about the principal evaluation system in this 
district?” Supervisors rated their agreement on a 5-point Likert-type scale. Differences between supervisors 
in PSI and other urban districts are statistically significant at the 5 percent level for all items except 
“...improving student attendance.” 


In the PSI districts, there was little evidence of conflict between supervisors’ dual roles as 
coach and evaluator. Principals and supervisors noted that increased ongoing support allowed 
supervisors to gain the trust, knowledge, and understanding of principals and their school context 
needed to evaluate principals accurately and fairly. 


D. Summary 


Principal supervisors in PSI and other urban districts spent their days in a similar manner. 
They spent a similar amount of time in schools, focused on similar topics, and engaged in high 
quality practices at about the same rates. The only major differences between supervisors’ 
practices in PSI and other urban districts were in principal evaluation. Perhaps because of the 
increased training and alignment of their role with principal evaluation, principal supervisors in 
PSI districts were more likely to agree that the work they engaged in with principals was more 
aligned with their district’s principal evaluation system. 
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The similarities in supervisors’ practices between PSI and other urban districts may reflect 
several trends and have several explanations. There is growing consensus in the field beyond the 
districts in the PSI about the role of principal supervisors, due in part to increasing 
professionalization of the role and the availability of guiding documentation, such as national 
standards. Principal supervisors in PSI districts also may have underrated their use of certain 
leadership practices as they gained deeper understanding of what these practices entailed through 
their PSI training. 
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Cee What do supervisors need to be successful? 


Wendied principal supervisors: As a leader of principals, what additional support do 
you need that would improve principal effectiveness and student achievement? 


Principal supervisors expressed a need for support in many areas. Supervisors in PSI 
districts made requests like those of supervisors in other urban districts, but often their 
requests were to continue receiving supports that were already provided. 


e Time. Supervisors’ number one request was for more time to coach principals and 
conduct principal professional learning meetings. 


@Q [I need] quality time spent with principals and more time for principals to have 
collegial discussions regarding instructional leadership and operations without 
interfering with their daily running of a school. 


Other urban principal supervisor 


e =© Assistance. Supervisors expressed a desire for better access to instructional and 
support staff whom they could deploy to schools to provide extra support to 
principals and to help them gather feedback about principal and teacher 
performance. 


@ It would be helpful to have a curriculum and instruction expert on my team (not a 
member of the curriculum and instruction department that | do not supervise and 
that is not accountable to my network’s goals, but an instructional expert who 
actually serves the schools in my network). 


PSI principal supervisor 


e Alignment. Supervisors stressed the importance of coherent goals, organization, 
norms, culture, and procedures across central office departments and between the 
central office and schools. Among other reasons, supervisors believed that central 
office alignment would allow them to more easily communicate with departments 
to obtain resources and support for principals. 


@ There needs to be a common language and theme within the district. Oftentimes 
departments are doing the same thing, but not talking with one another. 
Other urban principal supervisor 


Q We need a consistent vision and expectation for all departments to align and 
maintain focus on support of schools. 


PSI principal supervisor 
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e Collaboration. Many supervisors asked for opportunities to work with other 
principal supervisors in their district or to connect with supervisors in other 
districts. 


@ | would love to have a virtual place to collaborate, share ideas, and communicate 
with other principal supervisors outside of my district. 


Other urban principal supervisor 
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V. CONCLUSIONS 


Principal supervisors can play an important role in developing and supporting principals. 
National conferences, new principal supervisions standards, and other local efforts have 
promoted changes to the role to better support principals. At the same time, the supervisor role 
varies from district to district. Some principal supervisors may be the only such personnel in 
their district, with few resources available to support their development. Others might work in 
districts with clear definitions of the principal supervisor role and well-developed, role-specific 
trainings. 


The PSI prompted six districts to make specific structural changes to support revising and 
reframing the principal supervisor role. These included reducing supervisors’ span of control, 
providing role-specific training, and reorganizing the central office to better support principal 
supervisors’ work with schools. A comparison of key aspects of principal supervision across PSI 
and other urban districts can provide greater context for the changes that the PSI districts made 
during the initiative, highlighting the areas where PSI districts’ work aligned with broader 
national trends and the areas where it surpassed these trends. It can also provide a roadmap for 
other districts seeking to strengthen the principal supervisor role to better support principals as 
instructional leaders, by highlighting aspects of principal supervision that districts and states may 
find easier to change and aspects that may require more deliberate action and focus. 


A. Structures to support the revised principal supervisor role 


4) Principal supervisors in PSI districts had lower spans of control than those in other 
urban districts. Reducing principal supervisors’ span of control was a key goal of the PSI, 
intended to enable supervisors to spend more time working with each of their principals. By 
the end of the initiative in the 2017—2018 school year, supervisors in PSI districts oversaw 
an average of 13 principals, compared with an average of 16 for principal supervisors in 
other urban districts. However, spans of control in both sets of districts were considerably 
lower than the average span of control of 24 in urban districts in 2012 (Council of the Great 
City Schools 2013), suggesting that both PSI and other urban districts made structural 
changes to better support the principal supervisor role. 


@ Principal supervisors in PSI districts were more likely than those in other urban 
districts to oversee groups of principals organized by grade level. PSI districts put 
considerable thought into how best to assign principals to supervisors, and there were some 
differences in assignment approaches across PSI and other urban districts. Supervisors in 
PSI districts were more likely to oversee groups of principals organized by academic 
characteristics, such as school grade level, performance, or theme. Grouping principals this 
way can enable supervisors to narrow their focus to a set of content standards, curricula, and 
assessment data that is more homogeneous with their principal groups. In comparison, other 
urban principals were more likely to indicate that their principal groups were assigned 
according to feeder patterns, which required supervisors to work with principals across the 
K-12 spectrum. Such broad work could inhibit supervisors’ ability to develop specific 
expertise but might be necessary in districts with fewer principal supervisors. 
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3) Principal supervisors in PSI districts reported more role-specific training than did 
those in other urban districts and rated their training more highly. PSI districts focused 
on providing training and professional development that was specific and unique to the 
principal supervisor role, particularly in the first years of the initiative (Goldring et al. 2018 
and 2020). Supervisors in PSI districts clearly valued the training they received because it 
helped them hone their skills and, at the same time, build a sense of professional identity by 
working with fellow supervisors and coaches (Goldring et al. 2018). This type of role- 
specific training was more prevalent in PSI than in other urban districts. Furthermore, 
although their trainings emphasized some similar skills, supervisors in PSI districts rated the 
quality of their training more highly than did those in other urban districts. 


4) PSI districts were more likely than other urban districts to offer programs for new and 
aspiring supervisors. The PSI emphasized the importance of the central office in 
supporting and sustaining the new principal supervisor role. Central offices are responsible 
for cultivating and hiring principal supervisor candidates, supporting the development of 
current principal supervisors, and creating roles and structures to support supervisors’ work 
so they can meet the needs of principals and their schools. To ensure the sustainability and 
robustness of the new principal supervisor role, PSI districts worked to create plans and 
programs to fill principal supervisor vacancies with the best-prepared candidates and to 
deliver high quality training and mentorship to aspiring and new supervisors. Supervisors in 
PSI districts were more likely than those in other urban districts to report that (1) their 
district had a program to train aspiring supervisors, (2) their district had a mentoring and 
induction program for new supervisors, and (3) they themselves had entered their position 
through participation in a program for aspiring principal supervisors. 


(5) Principal supervisors in PSI and other urban districts had similar perceptions of 
central office support and structures. Early in the initiative, PSI districts realized that they 
needed to modify their central office structures and cultures to support and align with new 
principal supervisor roles. Overall, fewer than half of supervisors in PSI and other urban 
districts agreed that the central office understood their work, facilitated their work with 
principals, was organized to support principals, or scheduled meetings to maximize 
supervisors’ time in schools. Supervisors in PSI districts, however, were more likely to 
agree that they were involved in the deployment of instructional support staff to their 
schools. This gave supervisors the opportunity to allocate support to principals based on 
their specific needs. 


© Principal supervisors in PSI districts had more favorable views of their own evaluation 
systems than did those in other urban districts. PSI districts also invested in the 
evaluation and feedback supervisors received from their central office leaders. In many 
cases, these changes involved revising not only the expectations for principal supervisors’ 
performance, but also the roles of those who would be responsible for overseeing principal 
supervisors. Supervisors in PSI districts rated the quality of their own evaluations higher 
than did those in other urban districts on many dimensions, including the usefulness of 
feedback. 
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The role and practices of principal supervisors 


@ Principal supervisors in both PSI and other urban districts spent much of their time 


Cc. 


visiting schools focusing on instructional leadership. In 2015, the Council of Chief State 
School Officers released the first principal supervisor standards, which emphasized the 
importance of supervisors spending more time directly supporting principals (Council of 
Chief State School Officers 2015). These standards and other nationwide initiatives could 
have led to changing expectations for the role across urban districts that were similar to 
those in the PSI districts. Many PSI districts specified the amount of time supervisors should 
spend in schools each week or set a minimum number of visits supervisors should make to 
each school, and some implemented a differentiated support process called “tiering.” On 
average, supervisors in both sets of districts reported spending approximately half of their 
time on the job visiting schools and working with principals on instructional leadership. 
However, principal supervisors in PSI districts made slightly more frequent visits to both 
their most- and least-visited schools than did supervisors in other urban districts, although 
these differences were not statistically significant. This may have been a result of their 
smaller spans of control. 


Principal supervisors in both PSI and other urban districts used similar instructional 
leadership practices with principals, but supervisors in PSI districts were less likely to 
work on operational issues with principals. PSI districts revised their expectations for the 
principal supervisor role, including redrafting the job description and clarifying how 
supervisors should work with principals on coaching and feedback, data analysis, and 
classroom walkthroughs. PSI districts worked to ensure consistency of supervisor practices 
with principals. Supervisors in both PSI and other urban districts implemented similar 
instructional leadership practices with principals, but supervisors in PSI districts focused 
less than did those in other urban districts on budgeting and hiring. The PSI’s extensive 
training on instructional leadership and effective coaching may have led supervisors in the 
PSI districts to judge these practices according to higher standards, given their new 
understanding of what these practices entail. 


Principal supervisors in PSI districts had more positive views of their districts’ 
principal evaluation systems than those in other urban districts. PSI districts also 
focused on aligning their principal evaluation systems with principal supervisors’ work with 
principals. Supervisors in PSI districts expressed slightly more positive perceptions of their 
districts’ principal evaluation systems and were more likely to report that the systems held 
principals accountable for a variety of outcomes. 


Summary 


The PSI asked districts to make dramatic changes to the principal supervisor role, 


transforming a position that traditionally focused on managerial tasks to one dedicated to 
developing and supporting principals to be effective instructional leaders. At the same time, 
districts across the United States also were rethinking principal supervision, prompted by 
national conferences, new principal supervision standards, and other local efforts. However, the 
more intensive approach that the PSI districts followed led to some changes above and beyond 
those in other districts. 


A 


LEADING THE CHANGE: A COMPARISON OF PSI AND OTHER URBAN DISTRICTS VANDERBILT UNIVERSITY / MATHEMATICA 


The biggest differences between PSI and other urban districts, as reported by supervisors 
were in the structures to support the principal supervisor role. Compared with other urban 
districts, the PSI districts had lower spans of control, were more likely to provide unique and 
dedicated training for supervisors, and were more likely to offer programs for aspiring 
supervisors. In contrast, principal supervisors in PSI and other urban districts had similar 
perceptions of the support they received from the central office. Supervisors’ practices with their 
principals also were similar across PSI and other urban districts—supervisors reported spending 
more than half their time on instructional leadership and visiting schools. However, principal 
supervisors in PSI districts had more positive views of their districts’ principal evaluation 
systems than supervisors in other urban districts. 


Our finding of similar supervisor practices across PSI and other urban districts is consistent 
with two other recent studies, one focused on PSI districts and one on a broader set of urban 
districts, that examine how principal supervisors’ work with principals has evolved. The first 
study, Goldring et al. (2020), found that supervisors’ work with principals in PSI districts 
changed over the course of the PSI. Principals reported increased frequency of supervisor 
practices such as coaching, feedback, and data use. Principals’ perceptions of their supervisor’s 
effectiveness improved over the course of the initiative as well. Principals recounted how, over 
the course of the initiative, their relationships with their supervisors had improved and deepened 
because supervisors better understood their contexts and specific needs. The second study, 
Cochran et al. (2020), examined trends in principal supervision in a broader set of urban districts. 
It suggests that principal supervisors in urban districts have changed their practices to focus more 
on instructional leadership in recent years. Together these findings suggest that it is possible for 
districts to change the principal supervisor role to better support principals. 


Our findings are also consistent with prior research on districts as agents of organizational 
change, which distinguishes between bureaucratic changes and changes in professionalism 
(Hightower 2002; Johnson and Chrispeels 2010; O’Day 2002). These studies argue that top- 
down changes in bureaucratic structures, policies, resource allocation, and role definitions do not 
by themselves bring about professional changes, but rather create critical enabling conditions for 
these changes. For example, Johnson and Chrispeels (2010) concluded that strong district 
supports in structures, policies, and resources were important precursors to instructional 
improvement, but only when they supported pathways and conditions that promoted shared 
values, beliefs, and unity of mission between individuals in schools and the central office. In a 
study of how district conditions supported principal supervisors’ work with principals, Honig 
and Rainey (2019) found that certain internal district conditions, such as having a knowledgeable 
central office mentor, were beneficial in helping principal supervisors deepen their practices. 
They also noted that individual principal supervisors with stronger prior knowledge, who took it 
upon themselves to drive their own learning and protect their time, tended to make more growth 
in their practices—even compared with other supervisors in their district. These findings speak to 
the importance of district structural conditions, but also to the centrality of deep cultural shifts 
focusing on professional practice as keys to principal supervisor success. 


Other districts seeking to revise the principal supervisor role to better support principals can 
build on some of the work that we observed in the PSI districts. In particular, districts can help 
supervisors acquire a deep understanding of instructional leadership and learn how to develop it 
in principals. This requires developing a deep understanding of both high quality instruction and 
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instructional leadership practices. Without a clear understanding of high quality instruction, it is 
difficult for districts to determine and promote principal supervisor practices needed to support 
high quality instruction. To address this, some PSI districts worked with principal supervisors to 
develop and apply a common understanding of high quality instruction (for example, what they 
wanted students to learn and how they wanted teachers to teach). Some PSI districts also worked 
with principal supervisors to clearly specify expectations for instructional leadership. For 
example, they identified instructional leadership practices that principals should use in their 
schools (such as data-based decision making, observing instruction, and providing feedback to 
teachers). The need to develop a common understanding of high quality instruction and 
instructional leadership practices before principals and supervisors could dive deeply into 
instructional leadership development was an important finding from the study of the PSI. 


Deep changes to the principal supervisor role require clearly articulated expectations for the 
focus of the supervisor role, including strong expectations for consistency of practices; alignment 
with central office culture and capacity; and continued opportunities for principal supervisors to 
learn and refine practices. District leaders, practitioners, and researchers should continue to 
actively identify how and under what circumstances districts can promote these conditions to 
strengthen principal supervisors’ knowledge and practices. 
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APPENDIX A: SCALE DEVELOPMENT 


Table A.1. Training quality scale 


Specifically enhanced my capacity to develop principals’ instructional leadership 
Helped build a learning community with my fellow supervisors 

Provided opportunities to share specific practices with other principal supervisors in my district 
Was geared toward implementing district initiatives and programs 

Helped me understand district procedures 

Provided opportunities for me to receive feedback on my practice 

Addressed real challenges | face in my role 

Gave me opportunities to plan my work 

Was based on problems of practice | face in my role 

Provided opportunities for self-assessment of my skills 

Was engaging 

Provided me with actionable tools/resources | can use as a principal supervisor 
Facilitated my overall leadership 

Provided me tools to set goals for my own development 

Was interactive 

Taught me new knowledge and/or skills 

Was part of a sustained, systematic program for my development 

Stimulated my interest 

Allowed me to model practices | learned 


Helped me align my work with that of the other principal supervisors in the district 


Scale overall reliability (Cronbach’s alpha) = 0.96 


Note: The items are from the following survey question “Thinking about the district-sponsored training or 
professional development for principal supervisors you attended during the 2017-2018 school year and the 
summer before, to what extent do you agree or disagree with each of the following statements? ‘The 
professional development or training | attended...” (1 = Strongly disagree; 2 = Disagree; 3 = Neither 
agree nor disagree; 4 = Agree; 5 = Strongly agree). We created mean scale scores for training quality by 
averaging the 20 items at the supervisor level. We compared these mean scale scores with factor scores 
and found them to be virtually identical. We measured reliability using Cronbach’s alpha, a commonly 
accepted statistic for determining scale reliability. Coefficients above 0.90 are considered to have excellent 
internal consistency (DeVellis 2012). 
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